Virtualization of graphics resources

ABSTRACT

Graphics resources are virtualized through an interface between graphics hardware and graphics clients. The interface allocates the graphics resources across multiple graphics clients, processes commands for access to the graphics resources from the graphics clients, and resolves conflicts for the graphics resources among the clients.

This application is a continuation application of co-pending U.S. patentapplication Ser. No. 10/042,901, filed on Jan. 8, 2002.

RELATED APPLICATIONS

The present application is related to U.S. patent application Ser. No.10/043,018 (attorney docket number 004860.P2734) and Ser. No. 10/042,882(attorney docket number 004860.P2752), filed concurrently with andassigned to the same assignee as the present application.

FIELD OF THE INVENTION

This invention relates generally to computer graphics, and moreparticularly to virtualizing resources for computer graphics.

COPYRIGHT NOTICE/PERMISSION

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure as it appears in the Patent and TrademarkOffice patent file or records, but otherwise reserves all copyrightrights whatsoever. The following notice applies to the software and dataas described below and in the drawings hereto: Copyright© 1999-2002,Apple Computer, Inc., All Rights Reserved.

BACKGROUND OF THE INVENTION

A graphics kernel driver typically interfaces between graphics clientdrivers and graphics hardware to assign graphics resources to eachclient driver and to administer the submission of graphics commands tothe graphics hardware. Each client driver has explicit knowledge of thegraphics resources it is assigned and references the resources in itscommands using the physical address of the resources. As moresophisticated graphics features are developed, the demand for graphicsresources is ever increasing but the graphics resources are limited bythe graphics hardware and other system constraints. The assignedresources cannot be shared among clients because the graphics hardwareis not designed to handle resource contention among the clients.Additionally, the client drivers are required to manage their owninternal resource conflicts. For example, they must handle theirattempts to use more than available graphics memory.

SUMMARY OF THE INVENTION

Graphics resources are virtualized through an interface between graphicshardware and graphics clients. The interface allocates the graphicsresources across multiple graphics clients, processes commands foraccess to the graphics resources from the graphics clients, and resolvesconflicts for the graphics resources among the clients.

In one aspect, the interface is a graphics kernel that assigns anidentifier to a resource when allocated by a graphics client and theclient uses the identifier instead of an address for the resource whenrequesting access to the resource.

Because the native command structure for the graphics hardware isunaffected by the virtualization, neither the applications nor thehardware require modification to operate in conjunction with the presentinvention. Furthermore, because the virtualized resources appear asunlimited resources to the graphics clients, the clients can besimplified since, for example, they are no longer required tode-fragment or compact their assigned resources.

The present invention describes systems, methods, and machine-readablemedia of varying scope. In addition to the aspects of the presentinvention described in this summary, further aspects of the inventionwill become apparent by reference to the drawings and by reading thedetailed description that follows.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a diagram illustrating a graphics driver stack thatincorporates the present invention;

FIG. 1B is a diagram illustrating a system overview of one embodiment ofprocessing in the driver stack of FIG. 1A;

FIGS. 2A-B illustrate graphics command streams according to oneembodiment of the invention;

FIG. 3A-C illustrate processing of command buffers according toembodiments of the invention;

FIG. 4A is a flowchart of a graphics client driver method to beperformed by a computer processor according to an embodiment of theinvention;

FIG. 4B is a flowchart of a graphics kernel driver method to beperformed by a graphics processor according to an embodiment of theinvention;

FIG. 5A is a diagram of one embodiment of an operating environmentsuitable for practicing the present invention; and

FIG. 5B is a diagram of one embodiment of a computer system suitable foruse in the operating environment of FIG. 5A.

DETAILED DESCRIPTION OF THE INVENTION

In the following detailed description of embodiments of the invention,reference is made to the accompanying drawings in which like referencesindicate similar elements, and in which is shown by way of illustrationspecific embodiments in which the invention may be practiced. Theseembodiments are described in sufficient detail to enable those skilledin the art to practice the invention, and it is to be understood thatother embodiments may be utilized and that logical, mechanical,electrical, functional, and other changes may be made without departingfrom the scope of the present invention. The following detaileddescription is, therefore, not to be taken in a limiting sense, and thescope of the present invention is defined only by the appended claims.

In one embodiment, the present invention is integrated into a graphicsdriver stack 100 as illustrated in FIG. 1A. A graphics kernel driver 101interfaces between graphics client drivers 103, 105, 107, 109 andgraphics hardware 111 to virtualize limited graphics resources used bythe graphics hardware 111 and manage contention among the client driversfor the resources. The virtualized resources appear as unlimitedresources to the client drivers, which allows the client drivers to besimplified since, for example, they are no longer required tode-fragment or compact their assigned memory.

Graphics resources eligible for virtualization include any limitedresource used by the graphics hardware 111, such as graphics memory,either integrated in the graphics hardware 111 or allocated in systemmemory, GART (graphics address re-mapping table) entries, memoryapertures for accessing video memory or registers, specialized memoryareas for hierarchical depth buffers, among others. For the sake ofclarity, the virtualization of graphics memory is used as an examplethroughout, but the invention is not so limited.

Referring now to an exemplary embodiment shown in FIG. 1B, the kerneldriver 101 manages the allocation of memory among clients through avirtualization map 117, such as a range allocation table. Thevirtualization map 117 indicates how graphics memory is currentlyallocated, including which block a client is using.

An application 115 calls an OpenGL engine 113 through an OpenGL API(application program interface) 119 to create an image. The OpenGLengine 113, executing on the central processing unit (CPU) of thecomputer, determines how to divide the image processing work between theCPU and the graphics processor of the graphics hardware 111, and sendsthe commands to be processed by the graphics processor to the OpenGLclient driver 103 through a client driver API 121. The client driver103, also executing on the CPU, evaluates the commands and determinesthat it needs graphics memory to create the image. The client driver 103requests a block of memory from the kernel driver 101 through a kerneldriver API call 123. The kernel driver 101, executing on the graphicsprocessor, records the request in an entry in the virtualization map117, and associates an identifier with the entry. The kernel driver 101returns the identifier to the client driver 103 for use in all commandsthat access the memory block. Because the native command structure forOpenGL and the graphics hardware is unaffected by the virtualization,neither the application 115, the OpenGL engine 113, nor the hardware 111require modification to operate in conjunction with the presentinvention.

In one embodiment, the kernel driver 101 performs the actual physicalallocation of memory when the client driver 103 submits a command thatreferences the identifier. In another embodiment, the kernel driver 101physically allocates the memory upon receiving the allocation request.In either case, when all physical memory is already allocated, thekernel driver 101 pages a corresponding amount of data currently inmemory to a backing store and updates the virtualization map 117. Thekernel driver 101 uses the virtualization map 117 to determine how topage the data back into memory for subsequent processing. Details of thepaging are described further below in conjunction with FIG. 4B.

In one embodiment, the identifier is a “token” that represents thememory block and the client driver 103 creates tokenized commands bysubstituting the token for the memory address. When the client driver103 submits a tokenized command to the graphics hardware 111, the kerneldriver 101 extracts the token, finds the address of the memory blockrepresented by the token in the virtualization map 117, and replaces thetoken with the real address. When the tokenized commands are submittedas part of a standard graphics command stream, the kernel driver 101must parse the stream into its individual commands and evaluate most, ifnot all, the commands to determine which contain tokens. This can be aslow and expensive operation.

Therefore, in another embodiment, the client driver 103 formats thecommand stream as illustrated in FIG. 2B. A command stream 200 containsstandard commands 203, 205, followed by a tokenized command 207,followed by various other commands, and terminates with a tokenizedcommand 209. The stream 200 is prefaced with a “jump” packet 201 thatpoints to the first tokenized command 207 in the stream 200. Thetokenized command 207 contains another jump packet that points to thenext tokenized command in the stream 200, and so on until the last jumppacket in the stream is reached. The jump packets thus create a linkedlist of tokenized commands, allowing the kernel driver 101 to ignore thestandard commands in the stream 200 without having to evaluate eachcommand individually.

In one embodiment, the jump packets contain a packet type and an offsetvalue relative to the current packet. Assuming a command stream 210 asillustrated in FIG. 2B, the kernel driver 101 reads the first command inthe stream, which is a “start” jump packet 211. The kernel driver 101extracts the offset value from the start jump packet 211 and deletes thepacket from the stream. The kernel driver 101 uses the offset value tojump to the next jump packet 219, which is in the “load texture” command217. The kernel driver 101 extracts the next offset value and packettype from the jump packet 219. The packet type identifies the packet 219as a “texture” packet, indicating that the token 221 represents a blockof memory containing texture data. The kernel driver 101 replaces thetokenized command 217 with a valid graphics command 225 containing thememory address 223 corresponding to the token 221, and jumps to the jumppacket in the next tokenized command in the stream. The resulting stream220 received by the graphics hardware 111 contains “polygon” 213 and“change state” 215 commands unchanged from the stream 210 submitted bythe client driver 103, and a “load texture” command 225 as modified bythe kernel driver 101. Thus, the final processing of the command streamby the kernel driver only requires each jump packet to be read andwritten to and from memory while the majority of the command datagenerated by the client driver is not read or interpreted by the kerneldriver.

Alternate embodiments in which the jump packets are not embedded in thetokenized commands in the stream or are submitted as a separate streamassociated with the command stream are contemplated as within the scopeof the invention.

When a particular region of graphics memory requested by a currentclient driver has been reused by a previous client driver, the kerneldriver completes the use of the memory by the previous client driver,and prepares the resource for use by the current client driver. When thekernel driver processes a tokenized command, the graphics memoryreferenced by the token may be in one of two states: valid for immediateuse by the client driver or not. If the memory is valid for immediateuse, the kernel driver proceeds as previously described. If the memoryis not valid for immediate use, the kernel driver refreshes the currentclient's data by allocating a new region of graphics memory and page thedata into it. Before doing this however, the kernel driver submits allgraphics commands in the current client's command stream up to thecurrent jump packet to the graphics hardware before it starts allocatingthe new region of graphics memory for the current client because theprocess of allocation might result in the deallocation and paging ofgraphics memory previously referenced in the current command stream.Details of the refreshing of data are described further below inconjunction with FIG. 4B.

Command buffers are commonly used to hold the command streams frommultiple clients. As shown in FIG. 3A, as the client driver generatescommands, the CPU fills the appropriate buffer 301, 303. When a bufferis full, it is placed in a processing queue for the graphics hardware,and the CPU assigns another buffer to the client driver. It will beappreciated that when jump packets are used, the client driver loads thestart jump packet first in the buffer.

The command buffers allow multiple clients to create streamsasynchronously to each other. The command buffers also allow thegraphics hardware and the CPU to operate asynchronously, keeping bothbusy even though they typically operate at different speeds.

In one embodiment, the queued buffers are arranged as a linked list asshown in FIG. 3B. The contents of the buffers 301, 303, 305 are read bythe graphics hardware 111 as a linear stream of commands for executionin a serialized fashion, i.e., all the commands in one buffer areexecuted before the commands in the next buffer in the queue. Theserialized, linear execution by the graphics hardware 111 provides thekernel driver 101 with an memory management timeline to follow inprocessing the commands that reference graphics memory. After processingby the kernel driver, the entire command stream is valid for consumptionby the graphics hardware since the conflicts between clients due toreuse of memory have been resolved and the jump packets and tokenizedcommands have been replaced with valid graphics hardware commands.

In an alternate embodiment, the identifier for the memory blockallocated to the client driver 103 is the physical address of thememory. Because the client expects memory address to be unchanged untilit de-allocates the memory, the kernel driver 101 employs specialgraphics hardware features to manage the virtualization of memory. Inone embodiment, the kernel driver 101 uses graphics semaphores thatcause the graphics hardware to suspend processing of one buffer andswitch to processing another buffer, thus interleaving the processing ofthe command buffers from different clients, and creating multipleinter-dependent linear timelines as illustrated in FIG. 3C.

For example, assume client A places a command in buffer 307 thatreferences memory also used by client C. When the kernel driver 101reaches that command in buffer 307, it inserts a reference to semaphore313 before the command, effectively dividing the buffer 307 into commandsequences 311, 315. The graphics hardware 111 processes command sequence311 in buffer 307 until it reaches semaphore 313, which directs it toswitch to processing the next queued buffer 309. While the graphicshardware 111 is processing buffer 309, the kernel driver 101 pages theappropriate data back in and clears the semaphore 313.

Similarly, assume client B places a command in buffer 309 thatreferences memory also used by client D, so the kernel driver 101inserts a reference to semaphore 321 in buffer 309, creating commandsequences 319, 323. When the graphics hardware 111 reaches semaphore321, it determines that semaphore 313 is clear and resumes processingbuffer 307 at command sequence 315. Because the kernel driver 101 hascleared semaphore 321 by the time the graphics hardware finishesprocessing command sequence 315, the graphics hardware can now processcommand sequence 323.

Next, the particular methods of the invention are described in terms ofcomputer software with reference to a series of flowcharts. The methodsto be performed by a processing system constitute computer programs madeup of executable instructions illustrated as blocks (acts). Describingthe methods by reference to a flowchart enables one skilled in the artto develop such programs including such instructions to carry out themethods on suitably configured hardware (the processing unit of thehardware executing the instructions from machine-readable media). Theexecutable instructions may be written in a computer programminglanguage or may be embodied in firmware logic. If written in aprogramming language conforming to a recognized standard, suchinstructions can be executed on a variety of hardware platforms andinterface to a variety of operating systems. In addition, the presentinvention is not described with reference to any particular programminglanguage. It will be appreciated that a variety of programming languagesmay be used to implement the teachings of the invention as describedherein. Furthermore, it is common in the art to speak of software, inone form or another (e.g., program, procedure, process, application,module, logic . . . ), as taking an action or causing a result. Suchexpressions are merely a shorthand way of saying that execution of thesoftware by a machine causes the processor of the machine to perform anaction or produce a result. It will be further appreciated that more orfewer processes may be incorporated into the methods illustrated inFIGS. 4A-B without departing from the scope of the invention and that noparticular order is implied by the arrangement of blocks shown anddescribed herein.

Referring first to FIG. 4A, the acts to be performed by a computerprocessor executing a client driver method 400 that tokenizes commandsare shown. The client driver method 400 receives an image command (block401) and determines if graphics resources are required to process thecommand (block 403). If the necessary resources have not been previouslyallocated, the method 400 requests the resources from the kernel driver(block 405) and receives a token in return (block 407). The method 400creates the graphics commands to perform the image command at block 409.The processing represented by block 409 includes creating the jumppackets with the appropriate offsets and packet types, and inserting thejump packets and tokens in the commands. The particular packet typesused by embodiments of the invention are dictated by the command set ofthe underlying graphics hardware. One exemplary set of packet types,called “op codes,” for graphics memory are shown in Table 1. TABLE 1 OpCode Remarks kGLStreamStart Start the stream kGLStreamEnd Terminate thestream kGLStreamCopyColor Copy an image between two draw bufferskGLStreamCopyColorScale Copy an image between two draw buffers withscaling kGLStreamDrawColor Draw an image to the current draw bufferkGLStreamTexture0 Set the current texture object on texture unit zerokGLStreamTexture1 Set the current texture object on texture unit onekGLStreamTexture2 Set the current texture object on texture unit twokGLStreamTexture3 Set the current texture object on texture unit threekGLStreamNoTex0 Remove any texture from texture unit zerokGLStreamNoTex1 Remove any texture from texture unit one kGLStreamNoTex2Remove any texture from texture unit two kGLStreamNoTex3 Remove anytexture from texture unit three kGLStreamVertexBuffer Set the currentvertex object kGLStreamNoVertexBuffer Remove any current vertex object

If there is no existing command buffer (block 411), the method 400starts a new buffer (block 413) and inserts a start jump packet at thebeginning of the buffer (block 415) with an offset to the firsttokenized command in the buffer. Each graphics command is loaded in thebuffer (block 417) until all graphics commands are buffered (block 419)or the current buffer is full (block 421). If the current buffer is fulland more commands need to be buffered, the method 400 returns to block413 to start a new buffer.

Referring now to FIG. 4B, the acts to be performed by a graphicsprocessor executing a kernel driver method 430 corresponding to theclient driver method 400 are shown. The kernel driver method 430 isillustrated as two parallel processing threads, one that interfaces withthe client driver (starting at block 431) one that interfaces with thegraphics hardware (starting at block 451). It will be appreciated thatthe invention is not limited to such parallel processingimplementations.

When the method 430 receives an allocation request from a client driver(block 431), it determines if the requested amount of resource isavailable (block 433). If not, the method 430 pages out a sufficientamount of data belonging to another client (block 435). The method 430allocates the resource, including assigning a token and updating itsmemory management information, such as the virtualization map 117illustrated in FIG. 1B. The token is returned to the requesting clientdriver at block 439. The client driver method 430 waits until anotherrequest is received (block 441) and returns to block 431 to process thenew request.

When the client driver submits a buffer of commands to the graphicshardware for processing, the kernel driver method 430 extracts theoffset and type from the next jump packet in the buffer (block 451). Ifthe next jump packet is the first jump packet, i.e., a start jump packet(block 453), the method 430 deletes the start jump packet from thebuffer (block 461) and jumps to the jump packet defined by the offset(block 465) to continue processing. Otherwise, the method 430 uses thejump packet type to locate the token in the command and determines ifthe resource corresponding to the token has been reused (block 455). Ifso, the kernel driver method 430 refreshes the data required by thecurrent command (block 457). Because of the abstraction provided by thetoken, the kernel driver can page the data into a different availablegraphics resource or page out the data currently in the originalresource and page in the data required by the current command. The tokenis replaced with the address of the resource (block 459) and the jumppacket is deleted (block 461). If the current jump packet is the last inthe buffer (block 463), the method 430 waits for another buffer (block467) and returns to block 451 to process the new buffer. Otherwise, thenext jump packet in the buffer is processed.

In an alternate embodiment, the processing represented by block 437 is alogical allocation of the resource to the client driver and theprocessing represented by blocks 433 through 435 is not performed. Thekernel driver method 430 performs the physical allocation, and anynecessary paging, when it encounters the first tokenized command thatreferences the resource in the command buffer.

In one embodiment, the kernel driver method 430 uses system memory asits backing store for data that must be paged out of the virtualizedgraphics resources. The method 430 can request the CPU read the datainto system memory, or it can request the graphics hardware to write thedata to the system memory. The latter operation can be performedasynchronously with the CPU, but not all graphics hardware may be ableto perform the operation or there may be incompatibilities between thegraphics hardware and the CPU. When the operating system virtualizessystem memory, the operating system may further page the data to massstorage. It will be appreciated that once the data has been written tosystem memory, a virtual memory operating system may further page thedata to mass storage.

In one embodiment, what data to page into system memory is determined byvarious paging criteria, such as type of graphics resource, priority,and paging algorithm. Some resources, like graphics memory, are veryexpensive to page because the data contained in the graphics memoryoften must be copied into system memory. The priorities may be allocatedwithin graphics resources types. For example, texture objects generallyhave a lower priority than frame buffers when paging graphics memory.Other resources, like GART entries may be paged inexpensively becausethe paging only requires the modification of the GART table, i.e., nodata is actually relocated. Because the relative cost of pagingdifferent types of resources is quite different, different pagingalgorithms are used for each.

For example, when a client driver requests an allocation of graphicsmemory but there is not enough free contiguous memory to service therequest, all graphics memory resources owned by all clients arecandidates for paging. The first resources selected are owned by otherclients because there may be an arbitrarily long period of time beforethe other clients are run again. When considering graphics memory ownedby the requesting client driver, the kernel driver uses an algorithmthat dynamically switches from LRU (least recently used) to MRU (mostrecently used) based on whether or not the client driver isovercommitted in its texture usage. An overcommitted application is anapplication that uses more texture memory in rendering a single framethan can be supplied by the graphics hardware. When a client driver thatis not overcommitted runs out of graphics memory it is because some userinput has caused the client driver to render a new scene so the LRUalgorithm is used, based on the assumption that the least recently usedmemory resources may never be used again. When a client driver that isovercommitted runs out of graphics memory this means that it will do socyclicly every frame, so the MRU algorithm is chosen because an LRUalgorithm would result in every memory resource owned by the clientdriver being paged one or more times per frame.

GART entry paging is managed differently because the cost of changingGART entries is essentially unrelated to the size of the memoryresource. The first candidates for paging are GART entries that maynever be used again. For example, graphics memory texture objects eachhave a GART entry that was used to transfer the texture from systemmemory to graphics memory. Once the texture has been moved to graphicsmemory, the GART entry will never be used again unless the texture ispaged from graphics memory and then reloaded. Therefore, it is likelythat choosing such a GART entry for paging will have no performancecost. The remaining GART entries are categorized from highest to lowestpriority for paging, with the lowest priority assigned to the GART entryfor each client's command buffer, which must be mapped into GART for theclient driver to use the graphics hardware at all.

One of skill in the art will appreciate that other types of graphicsresources may have different algorithms for selecting which resourcesare candidates for paging that allow the resources to be transparentlymanaged with respect to multiple clients as described above for graphicsmemory and GART.

In one embodiment, the kernel driver method 430 uses a collection ofdata objects, each of which represents an allocated resource, as avirtualization map. The tokens identify the data objects within thevirtualization map. Each data object contains the address range for thecorresponding resource. When the data in the resource is paged out, a“dirty” flag is set and a pointer to the backing store holding the datais stored in the object. It will be appreciated that the layer ofabstraction between the client and the physical resources provided bythe token allows the data to be paged into a resource address differentthan it previously occupied without the client driver being aware of thechange.

The following description of FIGS. 5A-B is intended to provide anoverview of computer hardware and other operating components suitablefor performing the methods of the invention described above, but is notintended to limit the applicable environments. One of skill in the artwill immediately appreciate that the invention can be practiced withother processing system configurations, including hand-held devices,multiprocessor systems, microprocessor-based or programmable consumerelectronics, network PCs, minicomputers, mainframe computers, and thelike. The invention can also be practiced in distributed computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network.

FIG. 5A shows several computer systems that are coupled together througha network 3, such as the Internet. The term “Internet” as used hereinrefers to a network of networks which uses certain protocols, such asthe TCP/IP protocol, and possibly other protocols such as, for example,the hypertext transfer protocol (HTTP) for hypertext markup language(HTML) documents that make up the World Wide Web (web). The physicalconnections of the Internet and the protocols and communicationprocedures of the Internet are well known to those of skill in the art.Access to the Internet 3 is typically provided by Internet serviceproviders (ISP), such as the ISPs 5 and 7. Users on client systems, suchas client computer systems 21, 25, 35, and 37 obtain access to theInternet through the Internet service providers, such as ISPs 5 and 7.Access to the Internet allows users of the client computer systems toexchange information, receive and send e-mails, and view documents, suchas documents which have been prepared in the HTML format. Thesedocuments are often provided by web servers, such as web server 9 whichis considered to be “on” the Internet. Often these web servers areprovided by the ISPs, such as ISP 5, although a computer system can beset up and connected to the Internet without that system being also anISP as is well known in the art.

The web server 9 is typically at least one computer system whichoperates as a server computer system and is configured to operate withthe protocols of the World Wide Web and is coupled to the Internet.Optionally, the web server 9 can be part of an ISP which provides accessto the Internet for client systems. The web server 9 is shown coupled tothe server computer system 111 which itself is coupled to web content10, which can be considered a form of a media database. It will beappreciated that while two computer systems 9 and 11 are shown in FIG.5A, the web server system 9 and the server computer system 11 can be onecomputer system having different software components providing the webserver functionality and the server functionality provided by the servercomputer system 11 which will be described further below.

Client computer systems 21, 25, 35, and 37 can each, with theappropriate web browsing software, view HTML pages provided by the webserver 9. The ISP 5 provides Internet connectivity to the clientcomputer system 21 through the modem interface 23 which can beconsidered part of the client computer system 21. The client computersystem can be a personal computer system, a network computer, a Web TVsystem, or other such computer system. Similarly, the ISP 7 providesInternet connectivity for client systems 25, 35, and 37, although asshown in FIG. 5A, the connections are not the same for these threecomputer systems. Client computer system 25 is coupled through a modeminterface 27 while client computer systems 35 and 37 are part of a LAN.While FIG. 5A shows the interfaces 23 and 27 as generically as a“modem,” it will be appreciated that each of these interfaces can be ananalog modem, ISDN modem, cable modem, satellite transmission interface(e.g. “Direct PC”), or other interfaces for coupling a computer systemto other computer systems. Client computer systems 35 and 37 are coupledto a LAN 33 through network interfaces 39 and 41, which can be Ethernetnetwork or other network interfaces. The LAN 33 is also coupled to agateway computer system 31 which can provide firewall and other Internetrelated services for the local area network. This gateway computersystem 31 is coupled to the ISP 7 to provide Internet connectivity tothe client computer systems 35 and 37. The gateway computer system 31can be a conventional server computer system. Also, the web serversystem 9 can be a conventional server computer system.

Alternatively, as well-known, a server computer system 43 can bedirectly coupled to the LAN 33 through a network interface 45 to providefiles 47 and other services to the clients 35, 37, without the need toconnect to the Internet through the gateway system 31.

FIG. 5B shows one example of a conventional computer system that can beused as a client computer system or a server computer system or as a webserver system. It will also be appreciated that such a computer systemcan be used to perform many of the functions of an Internet serviceprovider, such as ISP 5. The computer system 51 interfaces to externalsystems through the modem or network interface 53. It will beappreciated that the modem or network interface 53 can be considered tobe part of the computer system 51. This interface 53 can be an analogmodem, ISDN modem, cable modem, token ring interface, satellitetransmission interface (e.g. “Direct PC”), or other interfaces forcoupling a computer system to other computer systems. The computersystem 51 includes a processing unit 55, which can be a conventionalmicroprocessor such as an Intel Pentium microprocessor or Motorola PowerPC microprocessor. Memory 59 is coupled to the processor 55 by a bus 57.Memory 59 can be dynamic random access memory (DRAM) and can alsoinclude static RAM (SRAM). The bus 57 couples the processor 55 to thememory 59 and also to non-volatile storage 65 and to display controller61 and to the input/output (I/O) controller 67. The display controller61 controls a display on a display device 63, such as, for example, acathode ray tube (CRT) or liquid crystal display, in accordance with thepresent invention. The input/output devices 69 can include a keyboard,disk drives, printers, a scanner, and other input and output devices,including a mouse or other pointing device. The display controller 61and the I/O controller 67 can be implemented with conventional wellknown technology. A digital image input device 71 can be a digitalcamera which is coupled to an I/O controller 67 in order to allow imagesfrom the digital camera to be input into the computer system 51. Thenon-volatile storage 65 is often a magnetic hard disk, an optical disk,or another form of storage for large amounts of data. Some of this datais often written, by a direct memory access process, into memory 59during execution of software in the computer system 51. One of skill inthe art will immediately recognize that the terms “machine-readablemedium” and “computer-readable medium” includes any type of storagedevice that is accessible by the processor 55 and also encompasses acarrier wave that encodes a data signal.

It will be appreciated that the computer system 51 is one example ofmany possible computer systems which have different architectures. Forexample, personal computers based on an Intel microprocessor often havemultiple buses, one of which can be an input/output (I/O) bus for theperipherals and one that directly connects the processor 55 and thememory 59 (often referred to as a memory bus). The buses are connectedtogether through bridge components that perform any necessarytranslation due to differing bus protocols.

Network computers are another type of computer system that can be usedwith the present invention. Network computers do not usually include ahard disk or other mass storage, and the executable programs are loadedfrom a network connection into the memory 59 for execution by theprocessor 55. A Web TV system, which is known in the art, is alsoconsidered to be a computer system according to the present invention,but it may lack some of the features shown in FIG. 5B, such as certaininput or output devices. A typical computer system will usually includeat least a processor, memory, and a bus coupling the memory to theprocessor.

It will also be appreciated that the computer system 51 is controlled byoperating system software which includes a file management system, suchas a disk operating system, which is part of the operating systemsoftware. One example of an operating system software with itsassociated file management system software is the family of operatingsystems known as Mace OS from Apple Computer, Inc. of Cupertino, Calif.,and their associated file management systems. The file management systemis typically stored in the non-volatile storage 65 and causes theprocessor 55 to execute the various acts required by the operatingsystem to input and output data and to store data in memory, includingstoring files on the non-volatile storage 65.

Virtualization of graphics resources has been described. Althoughspecific embodiments have been illustrated and described herein, it willbe appreciated by those of ordinary skill in the art that anyarrangement which is calculated to achieve the same purpose may besubstituted for the specific embodiments shown. This application isintended to cover any adaptations or variations of the presentinvention. The terminology used in this application with respect tographics is meant to include all environments that display images to auser. Therefore, it is manifestly intended that this invention belimited only by the following claims and equivalents thereof.

1. A computerized method of virtualizing graphics resources comprising:receiving, by a graphics kernel, an allocation request for a graphicsresource from a graphics client; allocating, by the graphics kernel, thegraphics resource to the graphics client; returning, by the graphicskernel, an address for the graphics resource to the graphics client;receiving, by the graphics kernel, a command from the graphics clientspecifying the address; and managing, by the graphics kernel, thegraphics resource.
 2. The computerized method of claim 1, whereinmanaging the graphics resource comprises: determining if the graphicsresource is available; and paging current data associated with thegraphics resource to a backing store if the graphics resource is notavailable.
 3. The computerized method of claim 2, wherein the currentdata to page is determining using paging criteria.
 4. The computerizedmethod of claim 3, wherein the paging criteria is selected from thegroup consisting of a type of graphics resource, a priority, and apaging algorithm.
 5. The computerized method of claim 4, wherein thepaging algorithm is based on usage of the graphics resource.
 6. Thecomputerized method of claim 1, wherein managing the graphics resourcecomprises: detecting a conflict if the graphics resource represented bythe token has been reused; and resolving the conflict.
 7. Thecomputerized method of claim 6, wherein resolving the conflictcomprises: inserting a reference to a graphics hardware semaphore beforethe command; paging the current data associated with the graphicsresource to the backing store; paging data for the graphics client fromthe backing store into the graphics resource; and clearing the graphicshardware semaphore.
 8. The computerized method of claim 1 furthercomprising: recording, by the graphics kernel, information about thegraphics resource in an entry in a virtualization map for use inallocating and managing graphics resources.
 9. A machine-readable mediumhaving executable instructions to cause a processing system to perform amethod comprising: receiving, by a graphics kernel, an allocationrequest for a graphics resource from a graphics client; allocating, bythe graphics kernel, the graphics resource to the graphics client;returning, by the graphics kernel, an address for the graphics resourceto the graphics client; receiving, by the graphics kernel, a commandfrom the graphics client specifying the address; and managing, by thegraphics kernel, the graphics resource.
 10. The machine-readable mediumof claim 9, wherein managing the graphics resource comprises:determining if the graphics resource is available; and paging currentdata associated with the graphics resource to a backing store if thegraphics resource is not available.
 11. The machine-readable medium ofclaim 10, wherein the current data to page is determining using pagingcriteria.
 12. The machine-readable medium of claim 11, wherein thepaging criteria is selected from the group consisting of a type ofgraphics resource, a priority, and a paging algorithm.
 13. Themachine-readable medium of claim 12, wherein the paging algorithm isbased on usage of the graphics resource.
 14. The machine-readable mediumof claim 9, wherein managing the graphics resource comprise: detecting aconflict if the graphics resource has been reused; and resolving theconflict.
 15. The machine-readable medium of claim 14, wherein resolvingthe conflict comprises: inserting a reference to a graphics hardwaresemaphore before the command; paging the current data associated withthe graphics resource to the backing store; paging data for the graphicsclient from the backing store into the graphics resource; and clearingthe graphics hardware semaphore.
 16. The machine-readable medium ofclaim 9, wherein the method further comprises: recording, by thegraphics kernel, information about the graphics resource in an entry ina virtualization map for use in allocating and managing graphicsresources.
 17. A processing system comprising: a processor coupled to amemory through a bus; a graphics processor coupled to the processorthrough the bus and associated with graphics resources; and a kerneldriver executed by the graphics processor to cause the graphicsprocessor to receive an allocation request for a graphics resource froma graphics client, allocate the graphics resource to the graphicsclient, return an address for the graphics resource to the graphicsclient, receive a command from the graphics client specifying theaddress, and manage the graphics resource.
 18. The processing system ofclaim 17, wherein the kernel driver, when managing the graphicsresource, further causes the graphics processor to determine if thegraphics resource is available, and page current data associated withthe graphics resource to a backing store if the graphics resource is notavailable.
 19. The processing system of claim 18, wherein the currentdata to page is determined using paging criteria.
 20. The processingsystem of claim 19, wherein the paging criteria is selected from thegroup consisting of a type of graphics resource, a priority, and apaging algorithm.
 21. The processing system of claim 20, wherein thepaging algorithm is based on usage of the graphics resource.
 22. Theprocessing system of claim 17, wherein the kernel driver, when managingthe graphics resource, further causes the graphics processor to detect aconflict if the graphics resource has been reused; and resolve theconflict.
 23. The processing system of claim 22, wherein the graphicsprocessor is associated with a graphics hardware semaphore, and thekernel driver, when resolving the conflict, further causes the graphicsprocessor to insert a reference to the graphics hardware semaphorebefore the command, page the current data associated with the graphicsresource to the backing store, page data for the graphics client fromthe backing store into the graphics resource, and clear the graphicshardware semaphore.
 24. The processing system of claim 17, wherein thekernel driver further causes the graphics processor to recordinformation about the graphics resource in an entry in a virtualizationmap for use in allocating and managing graphics resources.
 25. Agraphics system comprising: a graphics processor associated withgraphics resources; and a kernel driver executed by the graphicsprocessor to cause the graphics processor to receive an allocationrequest for a graphics resource from a graphics client, allocate thegraphics resource to the graphics client, return an address for thegraphics resource to the graphics client, receive a command from thegraphics client specifying the address, and manage the graphicsresource.
 26. The graphics system of claim 25, wherein the kerneldriver, when managing the graphics resource, further causes the graphicsprocessor to determine if the graphics resource is available, and pagecurrent data associated with the graphics resource to a backing store ifthe graphics resource is not available.
 27. The graphics system of claim26, wherein the current data to page is determining using pagingcriteria.
 28. The graphics system of claim 27, wherein the pagingcriteria is selected from the group consisting of a type of graphicsresource, a priority, and a paging algorithm.
 29. The graphics system ofclaim 28, wherein the paging algorithm is based on usage of the graphicsresource.
 30. The graphics system of claim 25, wherein the kerneldriver, when managing the graphics resource, further causes the graphicsprocessor to detect a conflict if the graphics resource has been reused;and resolve the conflict.
 31. The graphics system of claim 30, whereinthe graphics processor is associated with a graphics hardware semaphore,and the kernel driver, when resolving the conflict, further causes thegraphics processor to insert a reference to the graphics hardwaresemaphore before the command, page the current data associated with thegraphics resource to the backing store, page data for the graphicsclient from the backing store into the graphics resource, and clear thegraphics hardware semaphore.
 32. The graphics system of claim 25,wherein the kernel driver further causes the graphics processor torecord information about the graphics resource in an entry in avirtualization map for use in allocating and managing graphicsresources.
 33. An apparatus for virtualizing graphics resourcescomprising: means for receiving, by a graphics kernel, an allocationrequest for a graphics resource from a graphics client; means forallocating, by the graphics kernel, the graphics resource to thegraphics client; means for returning, by the graphics kernel, an addressfor the graphics resource to the graphics client; means for receiving,by the graphics kernel, a command from the graphics client specifyingthe address; and means for managing, by the graphics kernel, thegraphics resource.
 34. The apparatus of claim 33, wherein the means formanaging the graphics resource comprises: means for determining if thegraphics resource is available; and means for paging current dataassociated with the graphics resource to a backing store if the graphicsresource is not available.
 35. The apparatus of claim 34, wherein themeans for allocating the graphics resource further comprises: means fordetermining the current data to page using paging criteria.
 36. Theapparatus of claim 35, wherein the paging criteria is selected from thegroup consisting of a type of graphics resource, a priority, and apaging algorithm.
 37. The apparatus of claim 36, wherein the pagingalgorithm is based on usage of the graphics resource.
 38. The apparatusof claim 33, wherein the means for managing the graphics resourcecomprises: means for detecting a conflict if the graphics resourcerepresented by the token has been reused; and means for resolving theconflict.
 39. The apparatus of claim 38, wherein the means for resolvingthe conflict comprises: means for inserting a reference to a graphicshardware semaphore before the command; means for paging the current dataassociated with the graphics resource to the backing store; means forpaging data for the graphics client from the backing store into thegraphics resource; and means for clearing the graphics hardwaresemaphore.
 40. The apparatus of claim 33 further comprising: means forrecording, by the graphics kernel, information about the graphicsresource in an entry in a virtualization map for use by the means forallocating and the means for managing the graphics resource.